Papers with Prompt compression
TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learning (2025.findings-acl)
Copied to clipboard
Shivam Shandilya, Menglin Xia, Supriyo Ghosh, Huiqiang Jiang, Jue Zhang, Qianhui Wu, Victor Rühle, Saravan Rajmohan
| Challenge: | Existing prompt compression techniques rely on sub-optimal metrics such as information entropy or model it as a task-agnostic token classification problem that fails to capture task-specific information. |
| Approach: | They propose a task-aware prompt compression method that leverages existing Transformer encoders and a lightweight REINFORCE algorithm to ensure low latency requirements. |
| Outcome: | The proposed method improves task performance by 8% - 189% on three diverse and challenging tasks over state-of-the-art techniques while satisfying the same compression rate and latency requirements. |
Style-Compress: An LLM-Based Prompt Compression Framework Considering Task-Specific Styles (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prompt compression reduces inference time and costs while maintaining informativeness for different usage scenarios. |
| Approach: | They propose a framework that adapts a smaller language model to compress prompts for a larger model on a new task without additional training. |
| Outcome: | The proposed framework outperforms two baseline models in four tasks . iteratively generates and selects effective compressed prompts as task-specific demonstrations . |
From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have recently exhibited performance gains owing to a wide variety of prompting techniques, including Retrieval-Augmented Generation (RAG), Chain-of-Thought (CoT), and In-Context Learning (ICL). |
| Approach: | They propose a prompt compression method that captures the global context without compromising semantic consistency while detouring the necessity of pseudo-labels for training the compressor. |
| Outcome: | Empirical results show that the proposed method retains key contexts while reducing the prompt length by 80%. |
500xCompressor: Generalized Prompt Compression for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Prompt compression is important for large language models to increase inference speed, reduce computation cost, and improve user experience. |
| Approach: | They propose a method that compresses natural language contexts into a special token . they propose to reduce computations and memory costs by reducing the complexity . |
| Outcome: | The proposed method reduces computations and memory costs by 27-90% . it retains 70-74% and 77-84% of the LLM capabilities at high compression ratios . |